Planning and learning under uncertainty

Automated Planning is the component of Artificial Intelligence that studies the computational process of synthesizing sets of actions whose execution achieves some given objectives. Research on Automated Planning has traditionally focused on solving theoretical problems in controlled environments. In such environments both, the current state of the environment and the outcome of actions, are completely known. The development of real planning applications during the last decade (planning fire extinction operations (Castillo et al., 2006), planning spacecraft activities (Nayak et al., 1999), planning emergency evacuation actions (Muñoz-Avila et al., 1999) has evidenced that these two assumptions are not true in many real-world problems. The planning research community is aware of this issue and during the last years, it has multiply its efforts to find new planning systems able to address these kinds of problems. All these efforts have created a new field in Automated Planning called planning under uncertainty. Nevertheless, the new systems suffer from two limitations. (1) They precise accurate action models, though the definition by hand of accurate action models is frequently very complex. (2) They present scalability problems due to the combinatorial explosion implied by the expressiveness of its action models. This thesis defines a new planning paradigm for building, in an efficient and scalable way, robust plans in domains with uncertainty though the action model is incomplete. The thesis is that, the integration of relational machine learning techniques with the planning and execution processes, allows to develop planning systems that automatically enrich their initial knowledge about the environment and therefore find more robust plans. An empirical evaluation illustrates these benefits in comparison with state-of-the-art probabilistic planners which use static actions models. -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------La Planificación Automática es la rama de la Inteligencia Artificial que estudia los procesos computacionales para la síntesis de conjuntos de acciones cuya ejecución permita alcanzar unos objetivos dados. Históricamente, la investigación en esta rama ha tratado de resolver problemas teóricos en entornos controlados en los que conocía tanto el estado actual del entorno como el resultado de ejecutar acciones en él. En la última década, el desarrollo de aplicaciones de planificación (gestión de las tareas de extinción de incendios forestales (Castillo et al., 2006), control de las actividades de la nave espacial Deep Space 1 (Nayak et al., 1999), planificación de evacuaciones de emergencia (Muñoz-Avila et al., 1999) ha evidenciado que tales supuestos no son ciertos en muchos problemas reales. Consciente de ello, la comunidad investigadora ha multiplicado sus esfuerzos para encontrar nuevos paradigmas de planificación que se ajusten mejor a este tipo de problemas. Estos esfuerzos han llevado al nacimiento de una nueva área dentro de la Planificación Automática, llamada planificación con incertidumbre. Sin embargo, los nuevos planificadores para dominios con incertidumbre aún presentan dos importantes limitaciones: (1) Necesitan modelos de acciones detallados que contemplen los posibles resultados de ejecutar cada acción. En la mayoría de problemas reales es difícil obtener modelos de este tipo. (2) Presentan fuertes problemas de escalabilidad debido a la explosión combinatoria que provoca la complejidad de los modelos de acciones que manejan. En esta Tesis se define un paradigma de planificación capaz de generar, de forma eficiente y escalable, planes robustos en dominios con incertidumbre aunque no se disponga de modelos de acciones completamente detallados. La Tesis que se defiende es que la integración de técnicas de aprendizaje automático relacional con los procesos de decisión y ejecución permite desarrollar sistemas de planificación capaces de enriquecer automáticamente su modelo de acciones con información adicional que les ayuda a encontrar planes más robustos. Los beneficios de esta integración son evaluados experimentalmente mediante una comparación con planificadores probabilísticos del estado del arte los cuales no modifican su modelo de acciones

Integrating Planning, Execution, and Learning to Improve Plan Execution

Algorithms for planning under uncertainty require accurate action models that explicitly capture the uncertainty of the environment. Unfortunately, obtaining these models is usually complex. In environments with uncertainty, actions may produce countless outcomes and hence, specifying them and their probability is a hard task. As a consequence, when implementing agents with planning capabilities, practitioners frequently opt for architectures that interleave classical planning and execution monitoring following a replanning when failure paradigm. Though this approach is more practical, it may produce fragile plans that need continuous replanning episodes or even worse, that result in execution dead-ends. In this paper, we propose a new architecture to relieve these shortcomings. The architecture is based on the integration of a relational learning component and the traditional planning and execution monitoring components. The new component allows the architecture to learn probabilistic rules of the success of actions from the execution of plans and to automatically upgrade the planning model with these rules. The upgraded models can be used by any classical planner that handles metric functions or, alternatively, by any probabilistic planner. This architecture proposal is designed to integrate off-the-shelf interchangeable planning and learning components so it can profit from the last advances in both fields without modifying the architecture.Publicad

Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning

[EN] Observation decoding aims at discovering the underlyingstate trajectory of an acting agent from a sequence of observa-tions. This task is at the core of various recognition activitiesthat exploit planning as resolution method but there is a gen-eral lack of formal approaches that reason about the partialinformation received by the observer or leverage the distri-bution of the observations emitted by the sensors. In this pa-per, we formalize the observation decoding task exploiting aprobabilistic sensor model to build more accurate hypothesisabout the behaviour of the acting agent. Our proposal extendsthe expressiveness of former recognition approaches by ac-cepting observation sequences where one observation of thesequence can represent the reading of more than one variable,thus enabling observations over actions and partially observ-able states simultaneously. We formulate the probability dis-tribution of the observations perceived when the agent per-forms an action or visits a state as a classical cost planningtask that is solved with an optimal planner. The experimentswill show that exploiting a sensor model increases the accu-racy of predicting the agent behaviour in four different con-textsThis work is supported by the Spanish MINECO project TIN2017-88476-C2-1-R. D. Aineto is partially supported by the FPU16/03184 and S. Jimenez by the RYC15/18009Aineto, D.; Jiménez-Celorrio, S.; Onaindia De La Rivaherrera, E. (2020). Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning. Association for the Advancement of Artificial Intelligence. 11-19. http://hdl.handle.net/10251/178902S111

Computing programs for generalized planning using a classical planner

[EN] Generalized planning is the task of generating a single solution (a generalized plan) that is valid for multiple planning instances. In this paper we introduce a novel formalism for representing generalized plans that borrows two mechanisms from structured programming: control flow and procedure calls. On one hand, control flow structures allow to compactly represent generalized plans. On the other hand, procedure calls allow to represent hierarchical and recursive solutions as well as to reuse existing generalized plans. The paper also presents a compilation from generalized planning into classical planning which allows us to compute generalized plans with off-the-shelf planners. The compilation can incorporate prior knowledge in the form of auxiliary procedures which expands the applicability of the approach to more challenging tasks. Experiments show that a classical planner using our compilation can compute generalized plans that solve a wide range of generalized planning tasks, including sorting lists of variable size or DFS traversing variable-size binary trees. Additionally the paper presents an extension of the compilation for computing generalized plans when generalization requires a high-level state representation that is not provided a priori. This extension brings a new landscape of benchmarks to classical planning since classification tasks can naturally be modeled as generalized planning tasks, and hence, as classical planning tasks. Finally the paper shows that the compilation can be extended to compute control knowledge for off-the-shelf planners and solve planning instances that are difficult to solve without such additional knowledge.Anders Jonsson is partially supported by the grants TIN2015-67959 and PCIN-2017-082 of the Spanish Ministry of Science. Sergio Jimenez is partially supported by the grants, RYC-2015-18009 and TIN2017-88476-C2-1-R of the Spanish Ministry of Science.Segovia-Aguas, J.; Jiménez-Celorrio, S.; Jonsson, A. (2019). Computing programs for generalized planning using a classical planner. Artificial Intelligence. 272:52-85. https://doi.org/10.1016/j.artint.2018.10.006S528527

A Comprehensive Framework for Learning Declarative Action Models

[EN] A declarative action model is a compact representation of the state transitions of dynamic systems that generalizes over world objects. The specification of declarative action models is often a complex hand-crafted task. In this paper we formulate declarative action models via state constraints, and present the learning of such models as a combinatorial search. The comprehensive framework presented here allows us to connect the learning of declarative action models to well-known problem solving tasks. In addition, our framework allows us to characterize the existing work in the literature according to four dimensions: (1) the target action models, in terms of the state transitions they define; (2) the available learning examples; (3) the functions used to guide the learning process, and to evaluate the quality of the learned action models; (4) the learning algorithm. Last, the paper lists relevant successful applications of the learning of declarative actions models and discusses some open challenges with the aim of encouraging future research work.This work is supported by the Spanish MINECO project TIN2017-88476-C2-1-R and partially supported by the EU ICT-48 2020 project TAILOR (No. 952215). D. Aineto is partially supported by the FPU16/03184 and S. Jimenez by the RYC15/18009.Aineto, D.; Jiménez-Celorrio, S.; Onaindia De La Rivaherrera, E. (2022). A Comprehensive Framework for Learning Declarative Action Models. Journal of Artificial Intelligence Research. 74:1091-1123. https://doi.org/10.1613/jair.1.13073109111237

Learning action models with minimal observability

[EN] This paper presents FAMA, a novel approach for learning STRIPS action models from observations of plan executions that compiles the learning task into a classical planning task. Unlike all existing learning systems, FAMA is able to learn when the actions of the plan executions are partially or totally unobservable and information on intermediate states is partially provided. This flexibility makes FAMA an ideal learning approach in domains where only sensoring data are accessible. Additionally, we leverage the compilation scheme and extend it to come up with an evaluation method that allows us to assess the quality of a learned model syntactically, that is, with respect to the actual model; and, semantically, that is, with respect to a set of observations of plan executions. We also show that the extended compilation scheme can be used to lay the foundations of a framework for action model comparison. FAMA is exhaustively evaluated over a wide range of IPC domains and its performance is compared to ARMS, a state-of-the-art benchmark in action model learning. (C) 2019 Elsevier B.V. All rights reserved.This work is supported by the Spanish MINECO project TIN2017-88476-C2-1-R. Diego Aineto is partially supported by the FPU16/03184 and Sergio Jimenez by the RYC15/18009, both programs funded by the Spanish government.Aineto, D.; Jiménez-Celorrio, S.; Onaindia De La Rivaherrera, E. (2019). Learning action models with minimal observability. Artificial Intelligence. 275:104-137. https://doi.org/10.1016/j.artint.2019.05.003S10413727